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Abstract 
As in many industrialized countries, Japan carried out a deregulation of higher education around 
the turn of the 21“ century. This reform was based on New Public Management (NPM) principles, 
which should have been reflected in its most salient step: the incorporation of national universities 
in 2004. Although the way that the NPM principles were implemented in various countries was far 
from uniform, the Japanese version was no doubt quite peculiar. For example, from the outset, the 
management by objectives approach did not work well, due to deficits in the mechanism of ex-post 
evaluations. In addition, a roll-back of ministerial regulation emerged over time, under the guise of 
performance-based funding. As a result, universities rarely felt favored or promoted. Particularly in 
resource allocation among the institutions, they hoped for more transparency. It is assumed that this 
trend of re-regulation in the Japanese system will (in all likelihood) continue in the foreseeable 


future. 
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1. Introduction 

Around the turn of the 21* century, a substantial change occurred in the governance of 
higher education in the industrialized world (Hélscher, 2016). Specifically, in 
European countries, the concept of New Public Management (NPM) was introduced as 
new guiding principles of the higher education system (Austin & Jones, 2016). This 
change occurred against the backdrop of the structural transformation of the sector, 
particularly due to the increasing massification of higher education. Meanwhile, it 
became increasingly difficult for the government to enlarge its financial engagement 
with higher education institutions (HEIs). Since this situation also occurred in Japan, it 
is no surprise that the country carried out a deregulation of the sector. The centerpiece 
of the reform was the incorporation of national HEIs in 2004 (Oba, 2005; Kaneko, 
2009; Hanada, 2013). Until then, they were simply a part of the administrative 
structure of the education ministry. This reform, essentially inspired by the concept of 
executive agency of Thatcher-Britain, bestowed the national HEIs with the legal status 
of independent corporations, which included significant autonomy in financial and 
operational terms. 

Although many industrialized countries implemented NPM in higher education 
around that time, researchers have pointed out that there has been no single universal 
NPM regime (Boer, Enders, & Schimank, 2007, Broucke & De Witt, 2015). In other 
words, its concept modified according to each country’s social and cultural 
configurations. As a result, the NPM mode differed from country to country, including 
Japan. In this regard, the following research questions are raised: What changes 
occurred in Japan’s higher education system through the reform? How much did these 
changes conform to the NPM principles? Therefore, this study aims to answer these 
questions by focusing on their financial aspects. 


2. Structure of Japanese Higher Education 

First, it is important to provide an overview of Japan’s higher education system in 
order to clarify why we are examining national HEIs in this study. The Japanese higher 
education is borne by three sectors: national, local public (mainly prefectural and 
municipal), and private. Quantitatively, the private sector is overwhelmingly dominant. 
For example, its share in the number of HEIs is 77.4% and that in the number of 
students is 73.6%. By contrast, national HEIs (of which there are 86) account only for 
9.1% of the number of HEIs and 21.1% of the number of students, while such figures 
for local public HEIs are 14.9% and 5.3%, respectively (Ministry of Education, Culture, 
Sports, Science, and Technology [MEXT], 2018a). Although dwarfed by the number of 
private HEIs, the national institutions are the main pillars of the higher education 
system. In fact, many of the national HEIs have a considerable number of schools and 
institutes across a wide range of disciplinary fields. Notably, many of the Japanese 
institutions that emerge in world university rankings are the national ones. This is the 
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main reason why we are focusing on Japan’s national institutions. 

As for their finances, the national HEIs heavily rely on governmental funds, with 
the most important being the block grant. In 2019, its total sum for all of the national 
HEIs was JPY 1,097.1 billion (ca. USD 9.8 billion), accounting for 33% of their entire 
revenues (MEXT, 2020b). In return, the national HEIs are subject to control by the 
Ministry of Education, Culture, Sports, Science, and Technology (MEXT). Before the 
incorporation reform, control was carried out through minute ex-ante regulations. After 
2004, it was replaced with ex-post evaluations of performance based on the 
management by objectives concept. In other words, control of the universities was 
based on whether or to what extent the previous goals had been achieved. Thus, as part 
of the reform of 2004, the six-year business term was introduced, 1.e., all of the 
operations were planned and conducted within a cycle of six years. 

Prior to the beginning of the cycle, the HEIs determined what objectives they 
would achieve in each of their business fields of education, research, knowledge 
valorization, administration, and finances (medium-term objectives, MTOs), and what 
specific steps they would take to realize these goals (medium-term plans, MTPs). Both 
the MTOs and MTPs were submitted to the ministry for approval. At the end of the 
business term, a performance assessment was conducted, after which the outcomes 
were reflected in the block grant. In this case, good outcomes were honored with 
financial gains. 

This assessment is called the “National University Corporation Evaluation” 
(NUCE). Given the decisive significance of the ex-post evaluation in management by 
objectives, this evaluation scheme is the cornerstone of the six-year business term. In 
fact, the NUCE is a large-scale scheme equipped with considerable manpower and 
costs, since it addresses all of the MTOs and MTPs that the HEIs set for their various 
fields of activity. However, the MEXT does not solely rely on the NUCE for assessing 
the performance of HEIs. There are several other schemes, including categorization 
and uniform quantification. 

Regarding private and local public institutions, the state conducts no such 
performance evaluation. Indeed, the government provides private HEIs with public 
funds, but the sum is relatively small and mainly demand-based (e.g., the number of 
students and teachers).' Additionally, the performance assessment of local public HEIs 
is the responsibility of the prefectures and cities. 

As mentioned earlier, in post-2004 Japan, the basic NPM principles, namely, 
performance measurement and resource allocation based on its outcomes, are 
seemingly well in place at first glance. However, for a more proper understanding of 
the significance of the Japanese variant, we must focus on its details. 


" The subsidy for private institutions was JPY 584.3 billion (approximately USD 5.3 billion), 
accounting for 8.9% of their revenues (MEXT, 2020a). 
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3. NUCE 

The NUCE is not carried out by the MEXT itself, but by a committee set up for this 
purpose. For the core fields of education and research, the National Institution for 
Academic Degrees and Quality Enhancement of Higher Education (NIAD-QE) is 
commissioned to take over the evaluation because highly specialized knowledge and 
experience are required to review these fields (NIAD-QE, n.d.). As for the evaluations 
themselves, they are conducted through a peer review consisting of numerous 
university managers, professors, and researchers, with those from practical professions 
(e.g., business, journalism, and non-profit bodies) as external stakeholders. 

Despite being an ex-post evaluation, the NUCE is actually conducted twice during 
the six-year business term, i.e., during the fifth year and the year after the term’s end 
(i.e., the “seventh year”). The fifth-year review makes it possible for the HEI to draw 
on the evaluation results in time to prepare for the coming term. This is not possible in 
the seventh year because the next business term has already begun. As for the 
procedure, the NUCE follows the general rules of university evaluations. First, the HEI 
conducts its own evaluation of its activities and performance, and produces a report. 
Second, the reviewers, whom the NIAD-QE already selected, examine the report based 
on the criteria set by the agency. Subsequently, a site visit takes place (performed 
online in most cases) in which the reviewers interview the managers and teachers of 
the HEI. Finally, the reviewers draft the evaluation based on the results of both the 
document examination and the site visit. 

It is worth noting that, in contrast to the usual university evaluations (particularly 
those for accreditation), the NUCE’s foremost aim is resource allocation, i.e., deciding 
how to distribute the block grants among the national HEIs. For accreditation, the 
mission is to simply single out the ineligible institutions in absolute terms such as 
meeting the minimum requirements for higher education. Conversely, the NUCE 
compares the performance of HEIs and utilizes a single grading scale based on quality. 
In short, the NUCE is a relative evaluation. At the same time, however, the NUCE is 
based on individual achievement. That is, the quality of a HEI’s performance should be 
determined by to what extent the institution attained the MTOs established at the outset 
of the business term. However, this raises the following question: How is it possible to 
place various HEIs on a single grading scale when they are pursuing different 
objectives? 

In this regard, there are several devices built into the NUCE to cope with this 
difficulty. One of them is the stock-taking review of teaching and research, which is 
conducted as a sub-evaluation. Specifically, this sub-evaluation focuses on individual 
schools and institutions, and depicts an across-all-university perspective of the quality 
of the HEI’s operations. However, this does not solve the problem. First, it is 
questionable if the quality of teaching and research can be decided by one school. 
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Second, there is a logistical obstacle. Conducting a simultaneous review of all of the 
schools and institutions among the 86 national HEIs (totaling approximately 1,500) 
demands a considerable effort. If this should be carried out, then a meaningfully 
accurate examination will certainly fall victim to this approach. Third and most 
critically, the question will be raised on how the sub-evaluation can be grafted onto the 
NUCE, since the results gained from the inter-university comparative evaluation are, 
by definition, incompatible with those of the individual MTOs. 

Overall, the remedy of the sub-evaluation makes the NUCE’s architecture complex 
and inconsistent. Consequently, its results tend to be little differentiated. In the NUCE, 
judgment is made on a scale of several grades.” However, generally, reviews 
concentrate (instead of being scattered among the grades) on the middle ground with 
few ratings such as “Highly Excellent” or “Significantly Insufficient.” This 
concentration is especially striking in the domains of education, research, and social 
services and internationalization. As Table 1 shows, in the first as well as the second 
business term, the majority of the national HEIs are ranked in education and research 
as either “Excellent” or “Good.” 


Table 1 Distribution of Evaluation Grades in the NUCE 


The First Business Term Highly Significantly 
Excellent Good Insufficient 
(FY2004-09) Excellent Insufficient 
Education 0 | 0.0% | 11 {128% | 75 | 87.2%| 0 {0.0% | 0 | 0.0% 
Research 3 | 35% | 27 | 314% | 56 | 651%] 0 100%] 0 | 0.0% 
Social Services and 
2 +} 2.3% 36 | 41.9% | 48 | 55.8% 0 : 0.0% 0 | 0.0% 
Internationalization ! H 
Administration 25 | 29.1% | 47 | 54.7%] 13 | 151%] 1 112% | 0 | 0.0% 
EE 
The Second Business Term Highly Significantly 
Excellent Good Insufficient 
(FY2010-15) Excellent Insufficient 
Education 0 | 0.0% | 11 | 128%] 74 | 860%] 1 | 1.2%] 0 | 0.0% 
Research 5 | 5.8% | 13 | 15.1% | 67 | 779% | 1 {1.2% | 0 | 0.0% 
Social Services and 
1 3: 1.2% 21 } 244% 64 } 74.4% 0 : 0.0% 0 : 0.0% 
Internationalization H H i 
Administration 12 | 14.0% | 54 | 62.8% | 17 | 198%] 1 {1.2% | 2 | 23% 


Source: MEXT (2011, 2017) 


This all substantially undermines the NUCE’s relevance as a tool for 
ex-post-performance evaluation and, consequently, as a determinant of resource 
distribution. In fact, among the total sum of the block grants for the national HEIs 


> The number of grades is five in the first and second NUCE, increasing to six in the third. 
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(around JPY 1.1 trillion (ca. USD 10 billion)) in recent years, approximately JPY 3 
billion (ca. USD 27 million) or 0.3 % was earmarked for the NUCE’s results, while the 
MEXT set aside JPY 122.5 billion (ca. USD 1.1 billion) for other performance-based 
funding schemes (MEXT, 2020c). Meanwhile, the MEXT has not been transparent 
about how it redistributed the JPY 3 billion because there are no published rules 
regarding this process. 


4. Categorization and Uniform Quantification 

Considering the financial irrelevance of the NUCE feedback, it is safe to conclude that 
the incorporation of 2004 was effectively furnished by no concomitant 
performance-based funding. Yet, voices asking for this only grew louder both within 
and outside the government. On the one hand, it was argued that streamlining public 
funding for higher education was indispensable, in view of the deteriorating research 
power of Japanese HEIs in the international setting. On the other hand, there was the 
mounting budgetary deficit in the state finances. 

This pressure brought about a course change. First, in the entire framework of 
higher education financing, the weight of the baseline funding was curbed in favor of 
competitive project-typed financing. For example, the share of the block grants in the 
revenue of all of the national HEIs decreased from 46% in FY 2005 to 33% in FY 2019, 
while the third-party funds increased from 8% to 14% in the same period (MEXT, 
2020b). Second, and more importantly, the structure of the block grants was revisited, 
with more tangible components incorporated into performance-based funding. 

One of the initiatives for this purpose was the categorization of the national HEIs 
(MEXT, 2020d). Until then, the ministry had long adhered to the stance that all of the 
national HEIs were equal, despite the differences in size, composition of the 
disciplinary fields, and financial capabilities among the 86 institutions. In 2015, the 
MEXT introduced three categories of HEIs, after which each institution was asked to 
apply for one based on its profile and to choose goals. The categories were as follows: 


- Category 1: HEIs that focus on meeting the needs of the regional communities, 
while sharpening their profiles in education and research, particularly in the fields 
of their strength. 


- Category 2: HEIs that strive to promote excellence in education and research at the 
national and international level, particularly in fields of their strength. 


- Category 3: HEIs that strive to promote world-class activities in education, 
research, and knowledge- and technology-valorization, in competition with top 
universities abroad. 


In order to support the HEIs’ endeavors for profile sharpening, a central pool was 
established by top-slicing the block grant for each HEI. In this case, the sum 
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redistributed to each HEI was decided by a special committee consisting of 


approximately 10 experts, not by a quality assurance body such as the NIAD-QE. In 
FY 2016, when this initiative was launched, the pool totaled JPY 30.8 billion (ca. USD 
280 million) or 2.8 % of the total block grants of JPY 1,094.5 billion (ca. USD 10 
billion) (MEXT, 2015). However, this was not actually redistributed. For example, the 
ministry applied, for fear of sudden disruptions, a restraint mechanism so that the 


redistribution could occur within a range of 90%—110% of the original block grants. 


Even so, through these incentives, the categorization had controlling effects on the 


HEIs. 


Certainly, it was up to the individual HEIs to choose their respective category. Yet, 
the goals and indicators had to be approved by the ministry. More significantly, every 
indicator had to be quantifiable in order to be objectively measured. Table 2 shows 


how the goals and indicators were specified, based on an example from Mie University, 


a national HEI in central Japan. 


Table 2 Evaluation Results of the Categorization of Mie University (excerpt) 
Performance Target/Actual Score Score 
Indicators 2016 2017 2018 2019 | 2020 2021 (1) (2) 
Number of Target 
0 25 50 75 101 127 
local working (Person) 
adults Actual 
: 17 96 157 b a 
admitted (Person) 
Achievement 
- 384.0% | 314.0% 123.6% 
(%) 
“Mie Target 
0 420 840 1,260 | 1,680 2,100 
Revitalization (Person) 
Fantasista” Actual 
65 824 1,658 b a 
Qualification (Person) 
course Achievement 
aa - 196.2% | 197.4% 79.0% 
participation (%) 


Source: MEXT (2020e) 


As stated earlier, a committee of experts was appointed by the ministry to monitor 


the outcomes. Specifically, they examined each indicator every year from two 


perspectives: its ability to represent the university’s performance; and the university’s 
achievement of specific goals. They then turned the outcomes into scores, as shown in 


Table 2.> The scores were subsequently processed through formulas, after which the 


> In Table 2, Score (1) represents the indicator’s aptness, while Score (2) represents the university’s 
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redistribution ratios for each HEI were calculated. Despite some freedom granted to 
the HEIs, the procedure was (on the whole) based on a rigorous operation check by the 
ministerial administration. This left no sufficient room for discretion to the HEIs as 
agents, contrary to the management by objectives concept. This problem was 
compounded by the fact that the amount determined by the categorization initiative 
was separated from the lump-sum block grant. In other words, the money was only 
earmarked for the expenses that were related to the indicators in question. Moreover, 
demanding every goal to be quantified with numerical indicators and monitoring them 
on a yearly basis might have influenced the HEIs’ behavior in an undesirable direction. 
Excessive quantification and monitoring are neither suitable for activities that are 
primarily of qualitative in nature nor for those that require a long-term horizon. 
Consequently, it can discourage HEIs to include these types of activities in their list of 
goals, and hamper their involvement in long-term innovative activities. In addition, a 
formula can, by definition, only represent what the HEI already achieved in the past. 
Future-oriented innovative undertakings cannot be included in its scope. 

Regarding the categorization, the ministry seemed to have high expectations of it 
as a tool for steering HEIs. This was clearly seen from the sum set aside for the 
initiative. For instance, the initial volume of JPY 30.8 billion (ca. USD 280 million) in 
FY 2020 was no comparison with that for the aforementioned NUCE results of JPY 3 
billion (ca. USD 27 million). The categorization also seemed preferable to the ministry 
in that the initiative enabled it to quickly link the HEIs’ actual performance with 
funding. The monitoring results of categorization could thus be immediately reflected 
in the following year’s budget, while the NUCE was conducted at intervals of several 
years. 

Compared with the categorization, uniform quantification, the second initiative of 
performance-based funding, was even more regulatory (MEXT, 2018b). Introduced in 
2019, this initiative assessed the performance of all of the national HEIs based on 
uniform quantifiable metrics and formulas. The basic scheme was similar to that of 
categorization: The block grants were top-sliced to build a central pool, which was 
then redistributed to the HEIs according to each institution’s performance within a 
range of 90%—110%. The criteria for performance measurement, stipulated by the 
ministry, were as follows (MEXT, 2020d): 


- Graduates’ employment rate and percentage of bachelor’s degree recipients 
proceeding to graduate schools. 


- Those awarded doctoral degrees, compared with the intake of doctoral students. 
- Efforts made to improve education and quality assurance of learning outcomes. 
- Proportion of young researchers in the full-time faculty. 


success, based on a four-grade scale from the best as “a” to the worst as “d.” 
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- So-called “top 10% of citation” papers, compared with the block fund.* 

- Research output per full-time faculty member. 

- Grants-in-aid for Scientific Research awarded per full-time faculty member. 
- Commissioned and joint research projects per full-time faculty member. 

- Personnel and salary management reforms. 

- Diversity environment development. 

- Accounting management reforms. 

- Donations and other miscellaneous revenues per faculty member. 

- Facility management reforms. 


Each criterion was made objectively operational by including numerical metrics. Even 
the qualitative items were quantified by employing point systems. Meanwhile, 
controlling the criteria was carried out by a small committee set up by the ministry. 
These experts checked all of the metrics and calculated an assessment score for each 
HEI. 

The aforementioned list of criteria is quite striking. Formula funding is indeed a 
common practice for higher education financing in industrialized countries. Yet, it only 
addresses education and research. Apart from the shortcoming intrinsic to formula 
funding, there is another critical question: How constructive is it to gauge all of the 
HEIs on a single set of criteria? This will not be able to do sufficient justice to an 
individual institution’s strengths. It may also limit the diversity of higher education and 
eventually curtail the source of creativity in the university’s teaching and research. 

Generally, shortcomings concomitant to formula funding can be offset by 
combining it with performance/target agreements such as contract-based agreements 
between the government and individual institutions (Jaeger, 2009; Claeys-Kulik & 
Estermann, 2015). In contrast to the numerical approach of the formula, these 
agreements can make performance-based funding, in qualitative terms possible. They 
can also be framed in a tailor-made fashion, taking each HEI’s specificities/profiles 
into account. Moreover, they can include future-oriented innovative projects based on 
long-term visions. However, the MTOs, which should be comparable to these 
performance agreements, along with their assessment (i.e., the NUCE), do not live up 
to their missions and thus fail to serve as counterweights to quantification. 

Meanwhile, the expectations of the MEXT on uniform quantification have been 
high. For example, the amount earmarked for this initiative was JPY 70 billion (ca. 
USD 630 million) at the outset, JPY 85 billion (ca. USD 770 million) in the following 
year, and JPY 100 billion (ca. USD 900 million) in the third year. Likewise, the range 
of distribution was expanded from 85%-—115% to 80%-—120% (MEXT, 2020b; MEXT, 


* This index is only applied to Category 3 universities. 
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2020f). In this context, it is worth mentioning that, parallel to this tendency, the 
budgetary size of the categorization was downsized from JPY 25 billion (ca. USD 220 
million) in FY 2020 to JPY 20 billion (ca. USD 175 million) in the following year 
(MEXT, 2020f). Obviously, the policy weight shifted toward uniform quantification. 
This shift is likely to carry on, given that the voices supporting this initiative are only 
growing in government circles. 


5. Reactions of HEIs 

In order to consider how much the incorporation of 2004 corresponded to the NPM 
principles, we have thus far shed light on its structural and procedural aspects. Equally 
important is determining how the HEIs actually perceived the reform. To this end, 
there have been surveys conducted by the Center for National University Finance and 
Management (CUFM) (a NIAD-QE predecessor) between 2006 and 2014, in which the 
presidents and chief financial officers of the national HEIs were asked about the NUCE 
(CUFM, 2007; CUFM, 2010; CUFM, 2012; CUFM, 2015).° In this study, we mainly 
rely on the 2014 survey, which fell during the second business term (FY 2010-15). 


5-1. Effectiveness of Management by Objectives 

First, the survey inquired how effective they regarded the management by objectives 
implemented in the NUCE. The presidents of 84 of the 86 national HEIs responded on 
a five-grade scale ranging from 1 (“Ineffective”) to 5 (“Effective”), with 3 representing 
“Neutral.” The responses were then combined by institution type (see Figure 1° 


Fig. 1 Effectiveness of Management by Objectives 


3.20 3.40 3.60 3.80 4.00 4.20 4.40 


Former Imperial Universities (n=7) ———— rr 4.14 
Teachers’ Colleges (n=12) SSE SSS Sra 4.09 
Science and Engineering Colleges (n=13) SSS eae 4.31 
Humanities and Social Science Colleges (n=5) 4.20 
Medical Colleges (n=3) ———————— nd) 4.33 


Universities with Medical Schools (n=31 


Universities with No Medical Schools (n=10 


Graduate Schools (n=3 


Total (n=84 


> To date, there are no surveys of the HEIs’ views on categorization or universal quantification. 
° The J apanese national HEIs can be typologized into eight groups through criteria such as size, 
disciplinary and professional orientation, affiliation of the medical school/university hospital, etc. 
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Overall, the results were positive, with a majority of the presidents in favor. This 
differed from the prior expectation that the HEIs would not think highly of this 
management method because they would not welcome tighter control by the ministry. 
Interestingly, the positive attitude seems to be more distinct among the medical 
colleges, science and engineering colleges, and universities with medical schools, 
although no statistically significant differences were observed through a chi-square test 
of their cross-tabulation. This is probably because these colleges were more confident 
in the management by objectives approach. However, the opposite tendency was found 
among the graduate schools, teachers’ colleges, and universities with no medical 
schools. In fact, the only respondent who chose “Ineffective” was the president of a 
teachers’ college. 

It can be assumed that financial issues were behind these tendencies. It should be 
noted that the incorporation reform in 2004 had brought the national HEIs financial 
straits. With the backdrop of colossal deficits in the governmental budget, the ministry 
had prescribed an across-the-board cut of all block grants by 1% annually throughout 
the first business term (FY 2004-09). Additionally, the HEIs with affiliated hospitals 
were saddled with the so-called management improvement coefficient, which meant an 
automatic cut of 2% annually during the same period. This measure was a significant 
blow for every national HEI. However, perhaps the research-led HEIs managed to 
weather the hardship better because they could rely on other income sources such as 
third-party funds. In contrast, the colleges that were small in size and/or equipped with 
less external funding were squarely affected. This financial disparity among the 
institution types is presumably reflected in the fact that the teachers’ colleges and 
universities with no medical schools generally responded negatively. 


5-2. Satisfaction with NUCE 

Next, the NUCE itself was discussed in the survey. At the time of the 2014 survey, the 
HEIs had already received the NUCE results on their performance in the first business 
term. Specifically, the presidents were asked the following four questions: 


1. How satisfied were they with the results? 

2. How useful did they think the results would be for improving the management of 
their institutions? 

3. How accurately did they think the subsequent changes in the block grants reflected 
the NUCE results? 

4. How much did they want the results to be reflected in determining the block grants 
in the future? 


In the first question, the presidents were asked about each of the domains of 
education, research, social services and internationalization, and administration. Again, 
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there were 84 respondents who gave their input on a five-grade scale ranging from | 
(“Unsatisfied”) to 5 (“Satisfied”), with 3 representing “Neutral.” The responses were 
then combined by institution type (see Figure 2). 

This figure shows that the presidents were not particularly happy about education 
and research, whereas their satisfaction was high regarding administration. Behind this, 
there are two possible reasons. First, it should be noted that, generally, the outcomes 
can be more easily quantified in administration than in education and research. We can 
also assume that the evaluations by the NUCE reviewers were more objective in nature. 
Second, the HEIs presumably felt pressured to push through streamlining, especially in 
administration, as the aforementioned annual block grant reduction inexorably 
considered. 


Fig. 2 Satisfaction with the First NUCE Results 


3.00 3.20 3.40 3.60 3.80 4.00 4.20 4.40 4.60 


3.71 
Former Imperial Universities (n=7) 3.86 4.00 
4.29 
3.92 
Teachers' Colleges (n=12) eho 
3.83 
3.47 
~ bettas 7 3.69 
cience and Engineering Colleges (n=13) 3.54 
4.00 
320 
4 oe F ‘ “3 3.20 
umanities and Social Science Colleges (n=5) 3.20 
3.80 
3.33 
Medical Colleges (n=3) pe 
4.00 
3.49 
; es : ; = 3.45 
Universities with Medical Schools (n=31) 3.81 
3.94 
3.70 
a ale ; 7 3.90 
Universities with No Medical Schools (n=10) 380 
4.40 
4.00 
Graduate Schools (n=3) se 
4.00 
3.58 
Total (n=84) 38 = 


4.02 


@ Education B Research DSocial Services & International Relationship OD Administration 


As shown in Figure 2, there were differences among the institution types. However, 
they were not statistically significant after conducting a chi-square test of their 
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cross-tabulation. Interestingly, the teachers’ colleges and graduate schools were more 
content with the results of education and research than the other types. In contrast, the 
teachers’ colleges were least satisfied with administration of the four domains. 
Meanwhile, the humanities and social science colleges were least satisfied with 
education and research. These findings might be due to the lack of differentiation in the 
NUCE results, as mentioned in Section 2. For example, all of the teachers’ colleges had 
been graded in the first business term’s NUCE as “Good,” both in education and 
research. Some of their presidents probably felt favored by such lack of differentiation. 
Conversely, the humanities and social science colleges probably wanted a more distinct 
expression of their track record, instead of being lumped together with the others. 


5-3. Usefulness of the First NUCE Results for Improving Management 

In this question, there were 84 respondents who gave their input about each domain. 
Here, a four-grade scale ranging from | (“Useless”) to 4 (“Useful”) was employed (see 
Figure 3). 


Fig. 3 Usefulness of the First NUCE Results 


1.00 1.50 2.00 2.50 3.00 3.50 4.00 
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Overall, we can confirm a similar tendency as in the aforementioned satisfaction 
rates. The results of education and research were not that useful, while those of 
administration were more helpful. Interestingly, the findings about education were 
lower than those of any other domain. The reason might be that (as some respondents 
stated) the evaluation results were so general that almost all of the institutions were 
“Good.” This made it difficult for the HEIs’ leadership to draw meaningful suggestions. 
By contrast, in research and administration, the presidents were positive. This was 
surely based on the fact that the results were more differentiated in both domains, as 
Table 1 shows. 

It should be noted that, in this question, substantial differences were observed as 
statistically significant among the institution types, based on the chi-square test of their 
cross-tabulation (p <.01). In this regard, relatively small-sized institutions, such as the 
colleges of science, engineering, humanities, social and medical sciences as well as 
graduate schools (except for teachers’ colleges), estimated the NUCE’s usefulness less 
favorably than larger institutions. Some institutions, as their presidents stated, even 
decided to establish new organizations or reshuffle existing ones for better efficiency 
or for remedying their weaknesses pointed out in the NUCE. Among the relatively 
larger institutions, there were more positive views about the usefulness. For reference, 
the ratings of “Useful” at former Imperial Universities’ and universities with no 
medical schools in each domain are shown in Table 3. 


Table 3 Proportion of Responses of “Useful” in Two Institution Types 


Former Imperial Universities | Universities with No Medical Schools Total 
Education 42.9% 60.0% 26.2% 
Research 57.1% 60.0% 44.0% 
Social Services and 
57.1% 70.0% 47.6% 
Internationalization 
Administration 57.1% 70.0% 45.2% 


5-4. Linkage Between the First NUCE Results and the Block Grants 
Asked if they were satisfied with the way the first NUCE results were reflected in the 
block grants, the 84 presidents responded on a five-grade scale ranging from 1 
(“Unsatisfied”) to 5 (“Satisfied”), with 3 representing “Neutral” (see Figure 4). 

It is noteworthy that the average score for all of the respondents was 2.79. This 
indicates that the national HEIs saw no appropriate balance between their NUCE 


’ There are seven prestigious large-scale universities whose origins lay in the pre-war universities 
of imperial founding. They form the top tier of the Japanese higher education system. 
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results and the amount of the block grants. This discontent was particularly pronounced 
among the teachers’ colleges, humanities and social science colleges, and graduate 
schools, whose ratings were approximately 2.0. In fact, among these institutions, no 
president declared to be “Satisfied” or “Fairly Satisfied.” Interestingly, dissatisfaction 
among the teachers’ colleges was not conspicuous, compared with the others. 


Fig. 4 Satisfaction with the Block Grants Reflecting the First NUCE Results 
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The following comments by some of the respondents indicate the background of 
the HEIs’ reactions: 


- Some institutions end up favorably in the achievement measurement simply 
because they intentionally created flexible targets. 


- Regardless of the NUCE results, we suffer from the continuous cut of the block 
grant by 1% every year. Additionally, the ministry abolished other grants such as 
the one for promoting good practices in education. 


- Among the total amount of the block grant, the sum dependent on the NUCE 
results is almost negligible and falls short to incentivize managerial reforms. 


- Jt is unclear whether the NUCE results were reflected in determining the block 
grants in a fair manner. 


Although these complaints were particularly striking among the teachers’ colleges, 
similar negative views were (more or less) common across institution types. 


5-5. Desirable Influence of NUCE on Block Grant Reallocation 

As for the question of how much they believed the NUCE results should be reflected in 
determining block grant reallocation in the future, the 84 respondents chose one of the 
following answers: “Reflect More,” “Maintain the Status Quo,” “Reflect Less,” “No 
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longer Needed,” and “Others.” Figure 5 describes the cross-tabulation between the 
eight institution types and their choices. 

On the one hand, there was a group of institutions that wanted closer reflection. 
This group consisted of former Imperial Universities, science and engineering colleges, 
and humanities and social science colleges. They were apparently more confident in 
surviving in a contestable business environment created by performance-based funding. 
On the other hand, the teachers’ colleges, medical colleges, and graduate schools were 
of the opposite opinion. This finding indicates that the smaller institutions found 
themselves in a relatively disadvantageous position in resource reallocation. 


Fig. 5 Requests for How Much the NUCE Results Affect the Block Grants 
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In this context, some comments from the former Imperial Universities, science and 
engineering colleges, and humanities and social science colleges are telling: 


- Areallocation system would be appropriate that, after securing the basic budget for 
each HEI, would add a performance-based sum, which would be calculated on the 
point-adding principle. 

- Each HEI’s outstanding strategic efforts should be honored in this reallocation 
system, but the baseline should be separately secured. 


Interestingly, the stance of the teachers’ colleges, medical colleges, and graduate 
schools, despite their relatively precarious standing, did not differ much in this regard. 
Obviously, the principle of performance orientation had prevailed so much that it was 
difficult for them to raise outright objections to the seemingly irrefutable cause. They 
perhaps assumed that it might be beneficial to them, depending on how the principle 
was specifically applied. Note the following comments: 


- The government should encourage HEIs to be ambitious in setting strategic 
objectives. Additionally, performance evaluations should be based on their 
achievement of these goals by more transparent criteria. The results should also be 
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more fairly reflected in reallocation. These conditions are fundamental if the 
current system should be kept in place in the future. 


- Short-term assessments are unsuitable for performance in education and research 
because operations in these domains take considerable time to bear fruit. 


Regardless of what stance they took on the competitive environment, the HEIs 
basically agreed on how to remedy the shortcomings of the current reallocation system. 
First, they believed that the ministry should allow them to set challenging targets in the 
business term, after which a longer perspective is desirable for evaluating such 
ambitious objectives. Hence, the ministry should refrain from a hastily cutting off the 
block grants just because the HEIs failed to meet certain targets in one year. 

The issue of the overall burden was also addressed in the survey. It should be 
noted that the School Education Act prescribes every national HEI to achieve 
accreditation at the interval of no more than seven years. Additionally, the same act 
states that every institution is obliged to conduct an in-house performance evaluation 
on a regular basis and to make the results public. In the survey, there was a chorus of 
complaints. For instance, 53 of the 84 respondents stated that the preparations for the 
mandatory evaluations were not cost effective, and they requested that the ministry 
streamline the entire evaluation framework to ease the workload for the HEIs. 

Thus far, we have examined what the national HEIs thought of the NUCE and its 
financial consequences. We may state that the presidents’ views are moderate, neither 
strongly positive nor extremely negative. This is particularly true for the core fields of 
education and research. However, the fact that their satisfaction of the NUCE’s 
financial consequences lay under the average is worth emphasizing (see Figure 4). This 
indicates that the mechanism of performance measurement and resource allocation 
does not satisfactorily work in the eyes of the presidents. It also confirms that the NPM 
principles have not been sufficiently put into practice in the incorporation reform. 


6. Conclusion 

At the turn of the 21" century, Japan carried out a reform under the deregulation of 
higher education. In 2004, the reform bestowed the national HEIs with the legal status 
of independent corporations, and the business term system was introduced for ex-post 
evaluations of the HEIs’ performance. Thus, this study focused on the performance 
measurement and resource allocation of HEIs in Japan, and what the reform 
specifically brought about. Based on the findings, it is questionable how much the 
ideas of NPM were realized through this reform. 

First, a workable mechanism for ex-post evaluations was not established. The 
NUCE, which was supposed to play a key role, fell short of its expected mission due to 
the inconsistency of the structure and the ambiguity of the evaluation criteria. As a 
result, this evaluation scheme did not have a significant influence on the determination 
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of resource allocation. Meanwhile, the other two initiatives of performance-based 
funding (i.e., categorization and uniform quantification) met the NPM principles 
halfway at the most. Specifically, both approaches were too regulatory and did not 
leave enough room for discretion to the HEIs. Not only in their architecture, but also in 
their procedure, both were not characterized by management of objectives, but of 
operations. Striking is the strong reliance on detailed numerical metrics and the 
frequent monitoring of their progress. This aspect was particularly pronounced in 
uniform quantification. Additionally, the metrics were determined and prescribed to the 
HEIs by a group of experts appointed by the ministry who carried out monitoring and 
made judgments. This indicates that the ministry’s regulatory power remained 
substantially intact and the autonomy granted to the HEIs was limited. Moreover, both 
approaches lacked perspectives for future developments. 

Previous research has agreed that the realization of NPM principles does not mean 
a retreat of the state from controlling higher education. Instead, it is simply a new form 
of state control (Bogumil, Jochheim, & Gerber, 2015). Either way, one of the NPM 
tenets is to enlarge the agents’ freedom for more discretion. In this regard, it is safe to 
state that Japan’s post-incorporation system was not primarily characterized by NPM. 
Specifically, the regime of the minute ex-ante regulations prior to the reform 
essentially continued under a different guise. Why did the reform in Japan become 
stagnant halfway, despite the models in Great Britain and other European countries? At 
the time, under the buzzword of “small government,” quantitative downsizing of the 
public service sector had absolute priority, which put aside reviewing and reshuffling 
the competences of the ministerial bureaucracy (Osaki, 2011). 

In Japan, with the new business term of the national HEIs due to begin in FY 2022, 
the mode of higher education financing has become a topic of public discussion. Given 
the discussions thus far, the opinion in favor of stronger performance orientation 
continues to prevail. This is particularly important because the performance orientation 
in HEI funding is still fairly weak. Meanwhile, regarding the universal quantification 
initiative, changes in the block grants resulting from redistribution have remained 
within a range of —0.5% to +0.7%, even in FY 2020, when the initiative had been 
emphasized (MEXT, 2020b). 

Finally, it remains uncertain whether the trend in favor of stronger performance 
orientation will continue in the medium or long run. In this context, researchers have 
noted that NPM, when applied to public management, does not necessarily work as 
expected (Dunleavy, Margetts, Bastow, & Tinkler, 2006). As for higher education, 
there is already a discussion underway about the alternatives to NPM such as network 
governance or the public value model (Ferlie, Musselin, & Andresani, 2009; Broucker, 
De Witt, & Verhoeven, 2018). However, what will happen in Japan? Given the 
considerable backlog in accomplishing the NPM concept. Japanese higher education 
may still follow this approach or it will carefully watch the situations in European 
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countries and critically re-examine NPM as a whole. Anyway, the academic standard 
of the Japanese HEIs will greatly depend on which route they will select. 


* The views in this study are the authors’ and they do not represent those of the 
NIAD-QE, to which they belong. 
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